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FOREWORD 



This report, An Investigation of Sequential Search Algorithms , 
presents the results of a study done under Contract Number AF 19(628)- 
5989 for Decision Sciences Laboratory, Electronic Systems Division of 
the Air Force Systems Command, L.G. Hanscom Field, Bedford, Massa- 
chusetts. Dr. Ugo O. Gagliardi, ESRHT, was the Air Force program 
monitor. The investigation covered the period from 15 April 1966 to 
31 December 1966 and was presented as an interim report in January 1967. 
Appendix II was written by Drs. R.D. Johnson, Jr. , and S. Kneale of 
Operations Research Incorporated. 
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ABSTRACT 



Characterizations of sequential search processes and algorithms 
are developed. Representative sequential search algorithms are reviewed 
and interpreted within the framework of these characterizations in several 
fields including equipment diagnosis/ signal encoding, radar systems, 
coin weighing, and human decision processes. Conclusions and recom- 
mendations for application are presented. Appendixes include selected 
mathematical techniques in programming and information theoretic search 
procedures, analyses of scoring systems, and a bibliography. 
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SECTION I 



INTRODUCTION 



< 



1.1 This report documents the results of investigations of sequential 
search processes and their applications. The study establishes the unify- 
ing characteristics of a broad class of sequential search processes and 
algorithms that have been examined and developed in diverse fields and 
different contexts so that this body of formalisms can be applied in the 
future to problems that confront the Air Force. 

1.2 To accomplish this broad objective, the following subsidiary ef- 
forts were set forth: 

a. Studies and experimental investigations directed 
toward the consolidation and clarification of recent 
developments in coding theory, fault diagnostics, 
and search theory 

b. The generation of a conceptual framework sufficiently 
general to encompass existing search algorithms, 
and identification of directions in which future * 
efforts might most profitably be made. 

1.3 This document deals with the full scope of the defined investi- 
gation, which if completed, would increase the depth of analysis and the 
emphasis on applications of sequential search. 

Limitations in Scope 

1.4 To establish a viable scope within the prescribed effort, various 
initial limitations were explicitly identified. The processes under investi- 
gation were restricted mainly to sequential search, although at times non- 
sequential allocation of effort processes were considered to add back- 
ground perspective. The special class of sequential detection problems 
(looking for a signal in noise), so extensively dealt with in the statistical 
theory of communications, is also not considered primarily because they 
are usually not related to the classical sequential search processes. The 
additional special set of game theoretic search algorithms is also not ex- 
amined in detail although many such algorithms were encountered in pas- 
sing. 






Approach 

1.5 The defined tasks were approached using the concurrent defini- 
tion of a conceptual framework for generalized sequential search process- 
es and investigation of the specific search algorithms as an aid in refining 
this framework and determining the mathematical tractibility of the pos- 
sible solutions. This investigation was not intended to develop new 
algorithms for sequential search processes. However, as requested by 
ESD, an analysis of scoring functions was carried out. The results of 
this analysis are presented in Appendix II. 

Organization of Report 

1.6 Section II presents a conceptual framework for both sequential 
search processes and algorithms and discusses the distinction made be- 
tween the characterizations of the search processes and algorithms that 
were developed to optimize (or otherwise handle) these processes. Sec- 
tion III reviews and interprets the sequential search algorithms that have 
already been developed in various fields and different contexts. Section 
IV summarizes and presents conclusions and recommendations for further 
activities. Three appendixes are provided. Appendix I presents selec- 
ted mathematical techniques applicable to sequential search processes, 
and Appendix II describes the analytic work on scoring systems. Appen- 
dix III contains a bibliographic list of related papers, reports, and books. 
For the sake of more complete documentation the scope of the bibliography 
is somewhat broader than that of the study as defined. The list of cited 
references is given as the last section. 






SECTION II 

CONCEPTUAL FRAMEWORK FOR SEQUENTIAL 
SEARCH PROCESSES AND ALGORITHMS 



2.1 This section sets forth a unifying perspective for viewing se- 
quential search processes and algorithms that may have been developed 
independently and in different contexts. This development will provide 
a means of consolidating and clarifying recent developments in diverse 
fields to permit a more effective solution to problems confronting the 
Air Force. 

2.2 No claim is made for the uniqueness of this approach; in fact, 

a framework was sought that would be sufficiently general to include most 
sequential search processes and algorithms as special cases. The in- 
tent of the approach was to allow outstanding problems to be recognized 
within the context of this general framework as particular cases for which 
solutions may already be available. 

SEQUENTIAL SEARCH PROCESSES AND ALGORITHMS 

2.3 In this investigation, a distinction is made between sequential 
search processes and algorithms and the ways each can be characterized. 
The search process itself is characterized by a sequential decision tree, 
which represents a conceptual framework for the available system states 
and alternates inherent in the process. The algorithms themselves may 
be viewed as systems of boundary conditions and constraints imposed 

on the basic process. They embody the assumptions and restrictions 
that must be considered to structure the process into mathematically man- 
ageable proportions. 

CHARACTERIZATION OF SEQUENTIAL SEARCH ALGORITHMS 



2.4 A conceptual framework for sequential search algorithms is 
shown in Figure 1. That is, the algorithms relevant to this investigation 
can be characterized by a particular path through this figure. 

Multistage Processes 

2.5 The first dichotomous choice, at the top of Figure 1, breaks 
down search processes into single-stage (static) and multistage processes, 
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FIGURE 1 . PRELIMINARY SEARCH ALGORITHM FRAMEWORK 






Only the multistage process is considered in this investigation. This 
process is used synonomously with the sequential process. 

Nature of Search Space 

Z.6 In any search process, one or more search objects are located 

in one or more cells. The complete set of such cells may be said to con- 
stitute a search space. As shown in Figure 1, the search space itself may 
be characterized as discrete (and finite), continuous, or some combination 
thereof. In virtually all cases of practical interest, the search space is 
discrete and finite. If not, the mechanism for carrying out the search pro- 
cess is limited in its resolution so that the search space is, in effect, 
reducible to the discrete finite case. For example, in a radar search for 
a missile or aircraft, the vector coordinates (e.g. , position, velocity) of 
the search object may be described by a set of numbers that are infinitely 
dense (such as the position of a point on a line). But the radar's finite 
range, azimuth, and elevation resolution capability has the effect of limit- 
ing the search space to a finite number of discrete cells. 

2.7 The search space may also be conceptual as opposed to a physi- 

cal entity as in the radar search example. Consider the well-known coin 
weighing problem JLl±/ in which there are 1Z coins and a search is initiated, 
using an equal arm balance, for the one bad coin which is either heavier 
or lighter than all the other coins. Since there are 1Z coins and the bad 
coin may be either heavier or lighter, the search space consists of Z4 cells 
that do not exist as physical entities. The object of the search process, 
of course, is to isolate the cell describing the number of the coin and de- 
termine whether it is heavier or lighter than the others . 

Probability Distribution Over Search Space 

Z.8 Inherent in the search problem is the definition of some probability 

distribution over search space. Figure 1 only distinguishes between a uni- 
form and some other distribution at the initiation of the search. Several 
observations can be made in this regard. 

Z.9 Lacking other a priori information the assumption of a uniform 

distribution is usually made initially. There is some justification for 
this assumption, luA/ which states that the search object is equally 
likely to be contained in any one of the search cells. 



2.10 Clearly, as the search process itself continues, the probabil- 
ity distribution should depart from uniformity to the case in which the var- 
iance is forced to zero. This is reviewed in information theoretic terms 
as the sequential resolution of uncertainty, which is synonomous with the 
notion of decreasing the system entropy. In effect, this is the basis for 
using information theoretic approaches to search problems. 

2.11 Radar search, diagnostics, information retrieval, and numerous 
other types of search problems clearly illustrate the idea of a success- 
ively more highly peaked distribution and resolution of uncertainty as to 
the location of the search object. 

Nature of the Search Object 

2.12 A first-order partitioning of characteristics of the search object 
or objects distinguishes between stationary and nonstationary cases. If 
the object remains in its initial search cell over time or alternately over 
all stages of the search process, it is considered to be stationary. In 
the nonstationary case, the further distinction can be made between con- 
scious and nonconscious evasion. The former case implies a considered 
response, by the search object or an opponent, to the searcher's attempts 
to find or defeat it. Hence, game-theoretic formulations of search prob- 
lems fall into this category as well as many conventional duels in which 
the search for an optimum strategy is sought in the face of action by a 
competitive adversary. 

2.13 It is clear that the search problem becomes more difficult for 
the nonstationary object. Further, many different models for the behavior 
of the search object have been postulated, ranging from an attempted radar 
search for possible high velocity missiles to one where a search object 
shows up in a different cell on each trail, with complete independence 
from trial to trial. 

Measurement or Search Device 



2.14 The searcher has at his disposal some measurement device used 
to implement the search process. This is the mechanism used to decrease 
or resolve the uncertainty of locating the search object. 

2.15 Such a measurement device can be treated as either noiseless or 
noisy. In the former case, by definition, there are no errors of either the 
first or second type. In the context of radar detection, for example, such 
a device would always detect a target in a cell under surveillance and 









would never indicate its presence if, in fact, there were no target. Noise- 
less measurement devices are idealizations useful in the context of some 
types of search problems. The coin weighing problem mentioned above 
assumes a noiseless measurement device in that the equal arm balance can- 
not provide a false indication of balance or unbalance. 

2.16 The noisy measurement device, however, is often used and is the 
basis for much of the statistical theory of communications. In the radar 
example, it is more common to admit of nonzero false-alarm and failure-to- 
detect probabilities. Many search algorithms assume that there is some 
finite chance of not detecting the search object when observing the cell in 
which it is located and of detecting a nonexistent object in a cell. 

2. 17 Within the category of noisy measurment devices, the error prob- 
abilities can remain constant throughout the search or can change in some 
prescribed fashion, such as a function of the search space, time, stage of 
the search process, use of the measurement device, effort or resources de- 
voted to the search of each cell. Although these considerations might ap- 
pear to lead to insoluble problems, it is a fact that many search algorithms 
are based explicitly on such types of noisy devices. In the radar search, 
for example, the signal and noise power may vary with range so that each 
of the cells at different ranges has a different signal-plus-noise and noise- 
probability distribution leading to different failure-to-detect and false- 
alarm probabilities. 

Search Space Partitioning 

2.18 Another dimension to characterizing the sequential search algo- 
rithms involves the inherent ability of the searcher to partition the search 

space. This is important since it presents a basic constraint on the capa- 
bility of obtaining information about the location of the search object. 
This point can be illustrated by the coin weighing problem. 

2.19 Using an information theoretic approach (see Appendix I) shows 
that the initial uncertainty for 12 coins is of measure log (12)(2) = log 
24 and the potential information gathered from three weighings is 3 log 

3 = log 27. Since log 27 > log 24, it appears that three weighings are 
sufficient to find the bad coin. Although three weighings are, in fact, 
sufficient, the above condition is necessary but not sufficient. If the 
bad coin had to be found from among 13 coins, the potential information is 
still greater than the uncertainty, i.e., log 27 > log 26. However, the 
problem is not solvable in three weighings since it turns out that the 
maximum amount of information (log 3) cannot be obtained on each mea- 
surement. This results from the inherent inability to partition the search 






space, as reflected by the partitioning of the coins themselves, such 
that on each weighing the probabilities of the left side's being heavier, 
the right side's being heavier, and the two sides' balancing are equal. 

2.20 The same types of search space partitioning constraints are im- 
posed on radar search, diagnostics, information retrieval, and virtually 
all search problems. These constraints are often imposed jointly by the 
search object or space and the measurement device. For example, in the 
radar case, it is generally impossible to look at all combinations of 
search cells because of equipment limitations. It is also impossible to 
group test points arbitrarily in a diagnostic situation such that the mea- 
surement alternatives are equally likely. 

Allocation of Effort 

2.21 The next characteristic of sequential search algorithms, shown 

in Figure 1, involves the allocation of cost or effort over the search space. 
Many algorithms implicitly assume that this allocation is equal over all 
search cells; others allow this to be variable and, in fact, are seeking 
an allocation that maximizes or minimizes some figure of merit or meas- 
ure of effectiveness. 

2.22 As indicated previously, the probability of detecting a target in 
a particular cell may be an increasing function of the length of time or 
energy (effort) devoted to searching that cell. Based on the criterion that 
the overall probability of detection is to be maximized, an algorithm may 
be developed that will find the required time or energy allocations to 
each cell. Thus, the notion of allocation of effort (e.g. , time and cost) 
is intimately tied to the criterion (measure of effectiveness and figure 

of merit) on which the algorithm is based. Further, the allocation at 
each stage may affect the search environment by changing any or all the 
above characterizations. -^ 

Criterion 

2.23 The criterion for an algorithm is an explicit statement of the 
measure of effectiveness or figure of merit and the values that these 
should take. Figure 1 shows some of the more common criteria used: 
minimization of expected effort, minimization of quadratic effort, maxi- 
mization of detection probability, and maximization of information ob- 
tained. These are often stated together with subsidiary constraints 
such as achieving a maximum probability of detection, given a false 
alarm probability, or a total available effort (which can be devoted to the 
search process). 









SUMMARY OF SEARCH ALGORITHM CHARACTERIZATION 

2.24 From this discussion of sequential search algorithms, it should 
now be possible to cite for each existing or possible algorithm a path 
through Figure 1 for which that algorithm is characteristic / whatever the 
field of application or motivation for its development. At this point note 
that specific methods of solution, such as dynamic or linear programming 
and areas of application, have not been considered. This is done in 
Section III and Appendix I. 

2.25 The above characterization deals with the algorithms used 
for carrying out search processes and does not represent the search 
process itself. The remainder of this section discusses the latter case, 
i.e. , the characterization of the process and the insights that can be gain- 
ed by such a characterization. 

DECISION TREES AS SEQUENTIAL SEARCH PROCESSES 



2.26 A sequential search process is a process in which a sequence of 
decisions must be made in attempting to find something. After each de- 
cision, information is obtained that is relevant to the next decision. Con- 
sider first the case requiring a finite number of decisions and a finite num- 
ber of alternatives for each decision. (The case requiring an infinite num- 
ber of alternatives is not examined.) This process can be represented by a 
tree diagram as shown in Figure 2. Here the circles (or nodes) represent 
decision points, and the lines leading from them represent the alternatives. 
The square boxes represent the ends of decision sequences. Progress 
through the tree is made from top to bottom. 

2.27 The fact that circles and squares appear at the same horizontal 
level in the diagram does not imply that the decisions they represent 
must be contemporaneous; they are drawn this way for convenience, and 
their appearance at the same level means that they are at the same rela- 
tive positions in their sequences. 

2.28 In Figure 2 for example, there are nine possible paths through 
the tree, each path ending at a square box. A complete path through the 
tree is referred to as a "strategy." A strategy is a plan for carryinq out 
the search ; it involves allocating effort, partitioning the search 
space, and using the measurement device (as discussed previously and 
shown in Figure 1) in accord with some stated criterion . The square 
boxes, representing the ends of the various possible search sequences, 
correspond either to finding the object sought or to the decision to end the 




FIGURE 2 . A SIMPLE DECISION TREE 



10 












a. Decision Node With Single Alternative 
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b. Equivalent Decision Tree 
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FIGURE 3 . ILLUSTRATIVE DECISION TREES 
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a. Tree With Parallel Structure 




b. Equivalent Tree 




FIGURE 4. EQUIVALENCE OF PARALLEL STRUCTURED TREE 
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search process. Note that some strategies have more choices than others; 
thus the case is not excluded in which the number of decisions to be made 
depends on some of the decisions. 

Properties of the Decision Tree 

Z.Z9 It is assumed in this discussion that each node has at least two 
lines leading down from it. This is no real restriction, since if only one 
line leads from a node it means that in choosing to go to that node the de- 
cision maker has also committed himself to take the only available path 
from that node, so that he has really made the double decision of taking 
both these steps. Thus, it is assumed that in all such cases the two paths 
have combined into a single one, representing the single alternative. Fig- 
ures 3a and b illustrate this process. (The short horizontal line crossing 
certain paths indicates that the tree continues but is not reproduced here.) 
In Figure 3a, if path ZO is taken, path Z6 also must be taken at the next 
node. Figure 3b shows the replacement, with paths ZO and Z6 replaced 
by 35 and the node connecting them eliminated. 

Z.30 Assume also, at least temporarily, that every node (except the 
topmost ) has exactly one line leading into it. The conditions under which 
this assumption can be removed will be discussed later. The reason for 
having at least one line entering each node is obvious; that for restricting 
the number to at most one is not so obvious. In any event, this assump- 
tion does not really restrict the type of problem that can be represented by 
a tree (although it could make the tree have more branches than necessary). 
For if two different lines are followed by the same set of subsequent nodes 
and lines, the tree structure following can be attached to each of them, 
rather than meeting them at some node. Figure 4a is an example of two 
lines leading to the same set of subsequent nodes and lines, and Figure 
4b shows an equivalent tree. 

Z.31 Even in a relatively simple case requiring 10 decisions, each 

with 5 alternatives, there are 5 10 or about 10,000,000 different strategies 
possible. Thus, the tree diagram is usually a conceptual rather than a 
realized entity. Nevertheless, it does provide a basic framework for ex- 
amining sequential search processes. 

Measures of Effectiveness 

Z.3Z Deterministic Case. Each deterministic strategy leads to some 

final result, such as, !l the pay-off is $5, !l or !, 8 radars will be inoperable, " 
or "the object searched for is found after 7 trials." This result may be 
compared to a stated criterion. In each case a number is assigned to each 
possible strategy (actually to the outcome of that strategy). Even for the third case, 
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where there are only two possible outcomes (each attained by many dif- 
ferent strategies) M the object searched for is found" or "the object 
searched for is not found", the value "1" can be assigned to the first 
outcome and the value "0" to the second. Generally, the results pos- 
sible for the various strategies must, at the very least, ha\te an order 
relation established among them such that given any two strategies, 
either they are considered equivalent or one is better. Also, if this 
ordering relati.on is not to lead to logical contradictions, it is necessary 
for the ordering relation to be transitive; i.e. , if strategy A is better 
than B, and B better than C, then A is better than C. This is equiva- 
lent to saying that to each strategy there is assigned at least one number, 
measuring the effectiveness of that strategy. A simple measure of effec- 
tiveness will be defined to be a scalar, rather than vector, quantity. 

2.33 Probabilistic Case. It is also possible that the result of follow- 
ing a particular strategy can be expressed only in probabilistic terms* 
This will occur in cases for which the decision-maker has his choice of 
several probability distributions and the result of his choice is a sample 
from the distribution he has chosen. For instance, in the coin-weighing 
problem he may choose to weigh one coin against one, or two against two. 
The results of each of these choices are: (a) the coins balance, (2) the 
left side is heavier, (3) the right side is heavier. But, for each choice 

of how many coins to try, the probabilities of these results are different. 
Even if the measuring device is noisy, i.e. , the balance sometimes 
gives incorrect results, this case can still be fitted into the 
scheme if the probabilities of all the incorrect results are known. Of 
course, the cost of doing this is that the probability distributions from 
which the decision-maker chooses become more complicated. 

2.34 When the concept of a measure of effectiveness is added to the 
tree structure, the rationale for the restriction (in paragraph 2.30)— that 
only one path should lead into a node— is apparent. That is, even if the 
node and path structures following two different lines are identical, the 
measure for the total path, including the preceding nodes and lines, may 
be quite different. 

Examples of Measures of Effectiveness 

2.35 Coin-Weighing Problem. In the problem of finding the heaviest 
or lightest coin of 12 coins, with" an equal arm balance, a measure of 
effectiveness could be the number of coins that have been determined 

to be true coins or the number of measurements made to find a true coin. 
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This is distinguished from the objective or criterion, which is to find 
the bad coin and determine whether it is heavier or lighter in no more 
than three weighings . 

2.36 Frequency Assignment Problem . Suppose that 10 radars are to 
be operated in a region and that each has 15 frequencies to which it can 
be tuned. Suppose a frequency is needed for each radar so that the 
greatest number possible will operate satisfactorily. This can be thought 
of as a tree process in which the first step is to pick one of the 15 fre- 
quencies available for the first radar, then pick one from those available 
for the second. Thus, there are 15 10 possible strategies. A measure of 
effectiveness for a strategy in this case could be the number of radars 
operating satisfactorily if that set of frequencies is used. 

Types of Measures of Effectiveness 

2.37 Measures of effectiveness can be subdivided into two classes: 
those for which a number can be assigned for partial paths through the 
tree and those for which this cannot be done. If the measure function is 
the first kind, it is possible to get an answer to the question "How am 

I doing so far?". If it is not, then in a strict sense the problem is not 
one of sequential search or decision, since if no information is available 
(at each decision point) about how well we have done or the worth of the 
alternatives available, then we might as well have decided on the whole 
strategy at one time. That is, in a true sequential search or decision 
process, each time a new decision point is reached, some new informa- 
tion should have reached the decision-maker. In this context, many allo- 
cation of effort search problems-^ and their extensions are not really 
sequential. The second type of measure function will not be abandoned 
completely. However, for the present the first type is the only one to 
be considered. 

2.38 Measures of effectiveness of this kind— those that can be eval- 
uated at every node — are called separable. Thus, in the problem of the 
assignment of frequencies to radars, any time we have assigned frequen- 
cies to a subset of the total population it is possible to decide how many 
radars considered so far would be operating satisfactorily, given these 
frequencies. The measure function would then be separable. 

2.39 If the measure function is separable, it is always possible to 
replace it by an additive function, i.e. , by a function whose value at 
node N is the sum of its values up to the last previous node plus the 
value of the path from the last previous one to this one. Leaving out 
some unnecessary mathematical notation, the idea is that the number 
attached to a line should be the value of the measure function at its 
ending node minus the value of the measure function at its starting 
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F(17) = 26 

v(17, 33) = 9 




F(33) = 35 



FIGURE 5 . AN ADDITIVE FUNCTION ON A PATH 
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node. This is illustrated in Figure 5 where the value of the original 
(separable) measure of effectiveness was 26 at node 17, while it was 
35 at node 33. Hence the value 9 is assigned to the line from node 17 
to node 33. The value for the portion of a path up to a node will be the 
sum of the values for the lines along the path leading to that node. 

2.40 Thus, the general form for a decision tree with a simple and 
separable measure of effectiveness will be a sequence of nodes and con- 
necting lines and a number attached to each line. At the final box at the 
end of each complete path there is also a number, which is the sum of 
the numbers associated with the lines leading to it. Figure 6 is a typi- 
cal such tree. Note that the nodes have been numbered (for identifica- 
tion purposes). The lines can be identified by the numbers of the nodes 
they connect. The numbers alongside the lines represent the increment 
in the separable measure function that is attained if that line is used; 
the numbers under the final boxes represent the measure of effectiveness 
for the strategy that ends each path. 

2.41 Consider now the case for which the objective or criterion in 
the sequential search process is to find a strategy that maximizes the 
measure of effectiveness.* Of course, if the entire tree is drawn and 
all strategies evaluated, then the task of finding the best strategy is 
done by simply finding the largest measure value and tracing the strategy 
that led to it. However, in most practical cases the entire tree cannot be 
constructed because the number of possible strategies is enormous. 
Thus, the task to be faced is that of finding the best strategy without 
constructing and evaluating the whole tree. Naturally, the possibility 

of doing this depends heavily on the structure of the tree. Several 
techniques (algorithms) have been developed for solving these problems. 
In a gross sense, the more tightly the tree is structured the easier it is 
to find a solution or, equivalently, the larger the tree size that can be 
handled. Several of these techniques and the types of structures to which 
they apply are discussed later. 

2.42 Completely Replicated Trees. A completely replicated tree is 
one in which the structure following a node at any level is exactly the 



This is not necessarily always the case. As shown in Figure 1, the 
criterion might be minimizing an expected cost or maximizing a prob- 
ability. In the latter case, the measure of effectiveness may not be 
commensurable with the objective or criterion. 
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FIGURE 7 . COMPLETELY REPLICATED TREE 
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FIGURE 8. A PARALLEL TREE 




FIGURE 9 . PART OF THE PARALLEL TREE 
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a. Tree Equivalent to Parallel Tree of Figure 8 
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b. Tree Equivalent to Tree in Figure 10a 
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FIGURE 10. ILLUSTRATIVE EQUIVALENT TREES 
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same as that following every other node at that level. Thus, not only 
must the same decisions be available, but their values must be the same. 
Figure 7 is an example of a completely replicated tree. 

2.43 In trees of this kind, at any node the decision would be to take 
the next step along the path offering the largest value. Thus at node 1, 
take the line leading to node 2, since 6 > 1; at node 2, take the line lead- 
ing to node 4, since the value of 7 is the greatest attained. Finally, at 
node 4, take the line leading to terminal node 11. In case of ties, any 

of the maximum-value lines may be used. This technique finds the best 
path by a single run through the tree. 

2.44 The rationale behind this technique is simple. Since the struc- 
ture is replicated, the decision-maker knows that no matter what his 
choice is at any node the choices at the nodes that follow will be identical. 
Therefore, he might as well take the largest value available for this choice. 
The replicated tree structure will occur only in those cases where the ef- 
fect of a decision never depends on previous decisions. 

2.45 Parallel-Type Trees . In some sequential search processes, cer- 
tain decision nodes are followed by identical portions of the decision tree, 
with identical values for these decision lines. Under these conditions, it 
is allowable to simplify the tree structure by allowing more than one line 
to lead to a decision node. However, given this structure, the number 
under a final decision square no longer has a meaning because the same 
final square can be reached by more than one path. Figure 8 illustrates 
the situation. 

2.46 Note that there are three strategies that end at node 11 and three 
more at 12. However, the key to the matter can be obtained by looking at 
the partial tree structure up to node 6, as shown in Figure 9. No matter 
what decisions are made after node 6, the final sum will be the partial 
sum up to node 6 plus the partial sum for the rest of that strategy. But 

to get to node 6, the lines from 1 to 2 and from 2 to 6 give a sum of 8, 
which is greater than the sums for the other two paths. So, no matter 
how the decision process proceeds, the optimal process will never use 
the subpath from node 1 to node 3 to node 6, nor will it use the subpath 
from node 1 to node 4 to node 6. Thus, the original decision tree could 
be replaced by the one shown in Figure 10a. If the one-decision nodes 
are combined, Figure 10b results, in which nodes 3 and 7 are combined, 

2.47 This type of technique is used in finding the critical path 
through a PERT network. In the methods considered so far, the idea is 
to work down through the tree to each decision node that has more than 
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one line leading into it, then find the subpath leading to that node that 
gives the maximum subtotal. All other partial paths leading to that node 
are then dropped from further consideration. Because of the large number 
of such interconnections in a PERT network, it is usually possible to 
evaluate all the strategies available, after eliminating those that contain 
dominated parallel paths or subpaths, to find the optimum. 

2.48 Pseudo- Continuous Trees . The procedure outlined here applies 
in the case for which the same alternatives are available at all nodes at 
the same level. If the measure of effectiveness is separable and the incre- 
mental value for each alternative is the same, no matter what the preced- 
ing path, then this reduces to the completely replicated tree structure pre- 
viously considered. 

2.49 Suppose, as is often the case, that the choices at each node are 
really choices of some number, and that the value of this number has some 
meaning, not just as an identification procedure. For instance, in the pro- 
blem of assigning frequencies to radars, the choices available at any node 
are the frequencies available to the radar currently being assigned. Also, 
it seems evident that the total interference in the environment depends, 

in a somewhat continuous fashion, on the frequency assigned to the radar; 
i.e. , the total interference picture will not change much if the frequency 
of one radar is changed slightly. Under these conditions it is possible 
to talk about two strategies as being "close." Thus, if a strategy is 
thought of as an n-tuple of numbers, such as (ex , Qg , . . .en) where c. is 
the choice at node j, then if two such n-tuples agree at all positions but 
one, and the values at that position are close numerically, the two strate- 
gies can be called "close." The hope is that "good" strategies tend to 
cluster, so that if one finds a "fairly good" strategy, then a search around 
it is apt to find better ones. 

2.50 To apply this procedure, one must have an initial "fairly good" 
strategy. Sometimes this is just the result of a guess or of an empirical 
procedure to be outlined later. Suppose the initial n-tuple is (ex, Qg, . . .en) 
where each C* belongs to the class of numbers available for that decision. 



This is equivalent to imposing a metric on the class of n-tuples. The 
distance between strategy 1, given by (ci, c^, . . .en) and strategy 2, 

n 
given by (d^ d@, . . .d_) is d (Si, Sa) = £ | c x - d x | . 



n' 



i = 1 
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The technique starts by evaluating the n-tuple obtained by replacing Ci by 
its next higher value. If this improves, the value of the measure of effec- 
tiveness the next higher value is tried. This is continued until a value 
for the first number is tried which does not improve things. 

2.51 Next, the search for better values is started in a downward direc- 
tion from the initial choice. Of course, the standard of comparison now 

is the best value found so far. The search continues in this downward 
direction until a nonimproving change is tried. Then the same technique 
is tried with the second position of the n-tuple. This process is continued 
until the last position has been tried. 

2.52 The search can now be restarted by varying the first element 
again. (Since some other variables have been changed, this will not be 
a repeat of the previous search but will be different combinations 

of inputs.) The technique can be repeated as many times as desired. 
However, if a complete run through the n-tuple is made with no change 
in variables, the process might as well cease, since the attempts made 
now will be merely repetitions of the previous ones. 

2.53 If the measure function is separable, then an attempt may be 
made to obtain a good initial n-tuple by making a " one-step suboptimiza- 
tion' 1 trip through the tree. This would mean that at each node, the best 
line leading from it is taken (this technique is used in many human deci- 
sion processes). Of course, there is no guarantee that this procedure 
will find the optimal solution. Such a method is often called "heuristic. " 

Stochastic Sequential Search Processes 

2.54 A more general type of sequential search process is one in which 
the results of choices are not deterministic but are samples from known 
probability distributions. Thus, in making a choice at any decision node, 
the decision-maker does not know exactly what the result of his decision 
will be. Rather, he can only associate probabilities with each of the al- 
ternatives. This is similar, but not equivalent, to the probabilistic case 
discussed in paragraph 2.33 in which the measurement device provides in- 
formation that is not completely reliable. 

SUMMARY: FRAMEWORK FOR SEARCH ALGORITHMS AND PROCESSES 

2.55 This section has presented a tentative conceptual framework for 
both sequential search algorithms and processes. This framework is 
based upon the results of investigations to date. The processes are 
characterized by a decision tree describing the alternatives available at 
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each stage. The applicability of search algorithms is shown to be de- 
pendent on the structure and constraints placed on the decision tree. In 
summary, the following quotation from Singh 1/ is appropriate: 

M No computer, however rapid, could follow. . . through to 
success because the number of possible solutions. . .rises 
exponentially. It is the same with all other complex problems 
such as playing games, proving theorems, recognizing pat- 
terns, and so on, where we may be able to devise a recursive 
routine to generate possible solutions and a procedure to test 
them. The search fails because of the overwhelming bulk of 
eligible solutions that have to be tested." 

"The only way to solve such nonalgorismic problems by mech- 
anical means is to find some way of reducing ruthlessly the 
bulk of possibilities under whose debris the solution is buried. 
Any device, strategem, trick, simplification, or rule of thumb 
that does so is called a heuristic. ... In general, by limit- 
ing drastically the area of search, heuristics ensure that the 
search does terminate in a solution most of the time even 
though there is no guarantee that the solution will be optimal. 
Indeed, a heuristic search may fail altogether." 
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SECTION III 

REVIEW AND INTERPRETATION OF SEQUENTIAL 
SEARCH ALGORITHMS 



3.1 This section presents the methods used in the review and analy- 
sis of sequential search algorithms and some of the algorithms of par- 
ticular significance that have been examined. This review and analysis 
is neither exhaustive nor complete, but represents a report of progress to 
date, highlighting those areas of special significance and utility. Con- 
clusions and recommendations for further investigation are contained in 
Section IV. 

SORTING OF ALGORITHMS 

3.2 The basic approach to this investigation has involved the con- 
current tasks of developing a general conceptual framework for character- 
izing the processes and algorithms (Section II). This section investigates 
specific realizations of these processes and algorithms. Since the litera- 
ture supporting the latter is rather extensive, it was necessary to provide 
a mechanism for sorting and relating applicable algorithms so that com- 
mon characteristics could be recognized and interpreted. This mechanism 
is a simple numerical coding in conjunction with the characteristics estab- 
lished in Section II (Figure 1). Although the actual coding and subsequent 
analysis of all algorithms reviewed (which are presented in the bibliography 
and in this section) are not completed, it has been possible to draw a num- 
ber of significant inferences from the investigations. 

Numerical Coding 

3.3 The search algorithm characteristics are coded numerically in 
conjunction with the following scheme: 

First Digit— Type of Search Process 

1 . Single-stage 

2. Multistage 

Second Digit— Search Space 

1. Discrete 

2. Continuous 

3 . Discrete/continuous 
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Third Digit— Initial Probability Distribution 

1 . Uniform 

2. Other 

Fourth Digit— Search Object 

1 . Stationary 

2. Nonstationary 

2.1. Conscious evasion 

2.2. Nonconscious evasion 

Fifth Digit— Measurement Device 

1 . Noiseless 

2. Noisy 

2.1. Constant error probabilities 

2.2. Variable error probabilities 

Sixth Digit— Ability to Partition Space 

1 . Single-cell partitioning 

2. Multicell partitioning 

Seventh Digit— Allocation of Effort 

1 . Equal 

2. Variable 

Eighth Digit— Criterion 

1. Minimum expected effort (e.g. , cost and time) 

2. Minimum quadratic effort (e.g. , cost and time) 

3. Maximum probability of detection 

4. Maximum information per stage 

5. Maximum information per unit cost per stage 

6. Other 

This coding (itself a search problem) is not too cumbersome and allows 
for reasonable growth and reordering, if necessary. In addition, it was 
considered useful to characterize the methods used in the various search 
algorithms and their areas of application, as follows: 

Ninth Digit— Methods 

1 . Dynamic programming 

2 . Information theoretic 

3. Hypothesis testing 

4. Conventional decision theory 

5 . Other 
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Tenth Digit— Applications 

1. Human decision processes 

2. Equipment diagnosis 

3. Coding signals 

4. Radar systems 

5. Information retrieval 

6. Coin weighing 

7. Command and control 

8. General detection process 

9 . Others 

Dominant Algorithm Characteristics 

3.4 For those algorithms that have been coded in this manner, there 
is a clustering according to certain characteristics such as the natural 
tendency toward refinement and extension of problems with particular 
characterizations. For example, the search space is generally discrete; 
when it is not, the succession of solutions deals more appropriately 
with allocation of resources rather than strictly sequential search pro- 
cesses. The initial probability distribution of the search object among 
the search cells is almost always uniform. Both nonstationary and 
stationary models are considered together with both noisy and noiseless 
measuring devices. More often only single-cell partitioning is possible, 
and the variable allocation of effort models tends toward those using a 
noisy measurement device. The basic reason for this is that greater 
effort allocations (e.g., more time, power) are assumed to increase the 
probability of detection by providing greater margin between "signal" and 
"noise. " 

3.5 The dominant criterion for sequential search processes is clearly 
minimization of expected effort. In some cases, several criteria are 
used either separately or jointly to provide different solutions or solu- 
tions that are invariant with the selected criteria. Three nonexclusive 
areas seem to be prevalent concerning methods of solution: information 
theoretic, mathematical programming (linear, dynamic), and statistical 
decision theoretic, including both Bayesian and non-Bayesian approaches. 
The following paragraphs examine some of the more significant algorithm 
characteristics and inferences that can be drawn from them. 
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INFORMATION THEORETIC METHODS 

3.6 The information theoretic approaches use the conventional log- 
arithmic measures of uncertainty and are generally coupled with the 
criterion of minimizing expected effort, measured by such factors as 
cost, time, number of questions, and digits. In some cases, a measure 
of effectiveness is established, e.g., obtaining the greatest amount of 
information or the greatest information per unit of effort at each stage of 
the process with the intent that expected effort will be minimized or 
close to a minimum. 

3.7 Some of these methods are not on a firm mathematical foundation 
for several reasons. It is. well known that for many types of unconstrained 
decision trees, which is the more general case, the "optimum" choice 
from among all alternatives at each stage of the search process does not 
necessarily optimize the entire search process. This is the underlying 
root of the difficulty of handling generalized search and decision pro- 
cesses (cf. Section II: decision trees). 

Equipment Diagnostics 

3.8 Several authors, ^ apparently working together, have con- 
sidered the application of information theoretic concepts to diagnostic 
problems Jill-/ Their approach was to choose, at each step, from among 
all possible diagnostic tests, the one step that yielded the maximum 
amount of information per unit of cost. A figure of merit at each step was 
defined as the ratio of the entropy to the cost of performing the test. It 
was anticipated that such a sequential test procedure would be efficient 
in the sense of a low expected cost. Unfortunately, such a procedure 
provides no guarantees against a highly inefficient routine, as the fol- 
lowing counter-example demonstrates . 

3.9 Assume a piece of equipment, partitioned into four mutually ex- 
clusive cells, A, B, C, and D. Further, let the a priori probabilities 

of a defect (assuming only one defect) in these cells be 

p(A)=i p(B) = i, p(C) = p(D) =i 

Also, let the costs associated with locating the defect be C , Cu, C_, 

a c 

and C^ where C a is the cost of making a measurement on ceil A and de- 
termining whether the defect is in cell A. We are interested in locating 
the defect at least cost. This counterexample shows that the use of the 
figure of merit described above does not necessarily achieve this. 
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3.10 Let two methods of search be described by the following order 
of search: 

Method 1 — A, B, C 

Method 2 — B, A, C 

Note that a measurement on cell D is not necessary. The expected cost, 
using Method 1, is 



C + C K 
a b 



C + C, + C 
a b c 



a C 

b c 



Ci=— + — + 1 = C a+ T + — (3.1) 



The expected cost, using Method 2, is 

C u C + C u C + C,+ C C 

^r p a d a d c _ 3 _ c ,„ . 

C3 = — + i + " = C b + I C a + — (3.2) 



3.11 In both cases, the total uncertainty is 

H = - E p. log p. = H (I, i i i) = 1 .75 bits 



(3.3) 



However, using Method 1, we obtain 1 bit per measurement, and with 
Method 2, the information we obtain increases for each measurement. 
The expected number of measurements for Method 1 is 1.75, whereas 
for Method 2 it is 2. In both cases, of course, the expected information 
is equal to the total uncertainty of 1.75 bits. 

3.12 The figure of merit approach dictates starting with the measure- 

ment for which the figure of merit is a maximum, e.g. , choose 



max 



H 



H, 



H 



C ' C 
b c 



= max 



81 



.54 



(3.4) 



Of the two methods, 1 and 2, the figure of merit approach says choose 
Method 2 if 



or 



.81 1 

~~Z > "77 



1.23 n < c 
b a 



(3.5) 
(3.6) 
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In the region [1.23 C^ < C! < 2C, ] the figure of merit approach claims 

Da o 

that Method 2 should be used whereas actually Method 1 should be used 
in this region to obtain a lower expected cost. Therefore, the figure of 
merit approach does not necessarily minimize expected cost, and there is 
no reason to believe that this approach is an efficient one in a least- 
cost sense. 

The Coin Weighing Problem 

3.13 The coin weighing problem is a classical example of a search 
process and is examined in some detail here and in Appendix I. With 
12 coins, one of which is either lighter or heavier than the others and 
an equal area balance, it is possible to determine the bad coin in three 
weighings. The problem is to determine a search sequence that achieves 
this. 

3. 14 The greatest resolution of uncertainty is possible on the first 
measurement by weighing four against four r* If they balance, then two 
courses of action provide the most information on the second weighing: 
two of the remaining possible bad coins against one of the possible bad 
coins and a true coin, or three of the possible bad coins against three 
true coins. Either of these choices is satisfactory. If balance is not 
achieved on the first measurement, then 13 possible weighings are 
available, any one of which provides the greatest information on the 
second weighing. By following any one of these sequences, the last 
weighing is trivial. 

3. 15 The point here is that an information measure (entropy) is used 
as the measure of effectiveness at each stage of the process in the hope 
that by obtaining the greatest resolution of uncertainty at each stage, 
the overall uncertainty can be resolved in three (or a minimum number 

of) weighings. This is basically a heuristic technique since, even the res- 
olution of the greatest uncertainty at one stage, in the general case, 
may lead to a set of alternatives that cannot be partitioned so that 
the overall process is accomplished in a minimum number of stages. 

Sequential Search and Coding 

3.16 Various coding algorithms bear direct relations to sequential 
search processes such as those described above. For example, consider 
the binary coding of the message set [m ] below: 

00 where P(mi) = jt 



m x 



m 2 


01 


m3 


10 


nu 


11 



P(m 2 ) = i 
P(m3) = £ 
P(m<i) = & 
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FIGURE 1 1 . RELATIONSHIP BETWEEN BINARY CODING 
AND A SIMPLE SEARCH ALGORITHM 
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FIGURE 12. RELATION BETWEEN SHANNON-FANO CODING 
AND A SIMPLE SEARCH ALGORITHM 
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The average length (L)of a message is two digits and the efficiency of 
the coding scheme is defined as 

Efficiency = H fe' J*-!* t) = 7 / 4 = 87.5 percent (3.7) 

L log D 2 log 2 K x ' 

where H (•) = message entropy 

D = number of symbols in the coding alphabet . 

3.17 This coding scheme implicitly represents a search algorithm. 
Consider a search for one of the four messages that is carried out by 
receiving answers to dichotomous questions. The zero for the first 
digit for both mi or m s can be interpreted as the question "Is the message 
either mx or m s ? " . This process continues (see Figure 11) until the mes- 
sage is determined with certainty after two questions. 

3.18 Although the search algorithm represented by the conventional 
binary coding resolves all uncertainty with two dichotomous questions, 
another code can be found which, on the average, resolves the uncer- 
tainty in fewer than two questions. 

3. 19 Consider the coding which partitions the set of messages into 
two equally likely groups each time an alphabet symbol is assigned. 
Such a code, sometimes referred to as the Shannon-Fano encoding pro- 
cedure — =/, is 

mi where, as before, p(m x ) = 4 

m s 10 p(m s ) = 4 

m 3 110 p(m 3 ) = 4 

m 4 HI p(m 4 ) = 4 

3.20 The average length of this code (L) is I4 digits, which leads to 
a coding efficiency of 100 percent. This code reflects the search al- 
gorithm shown in Figure 12. Note that the code for each message is 
obtained by following each branch of the tree and adding coding digits 
as required by the decision at each junction. 

3.21 If we accept the minimization of the expected number of 
questions or code length as a criterion for effectiveness of the search 
strategy, then the Shannon-Fano technique is more efficient than straight 
binary encoding. However, the variance is zero for the latter case and 
nonzero for the former. For example, the probability is one quarter that 
three questions will have to be asked to resolve the uncertainty. This 
simple example shows the classical differences between using first and 
second moments as criteria for effectiveness of a search algorithm. 
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nnonrFano technique was extended to 
irrr analogous to a search process 



3.22 Although it is true that the Shannon-Fano encoding procedure 
provides a smaller average length code than the binary encoding, it 
is not the most efficient procedure. The Huffman code H* is, in 
general, a code that minimizes the average code length. Actually it 
is a minimum redundancy code with the irreducibility property. 

3.23 These coding examples illustrate the rationale for distinguish- 
ing between a search process and a search algorithm and the formal 
correspondence between coding and search theory. Figures 11 and 12 
present particular algorithms for isolating a message. Each particular 
algorithm is, in effect, a path through a decision tree that represents 
the set of all possible algorithms for isolating the search object. 

Progression of Noiseless Coding Algorithms 

3.24 The above example shows the relationship between minimum 
redundancy coding for the discrete noiseless channel and a search pro- 
cess. The progression of such coding algorithms started with Shannon 
-=■2/ and Fano JLi/ for near-optimum codes for which all code symbols 
have equal cost. As indicated above, Huffman i=/ developed a com- 
binatorial algorithm that does yield an optimum code (or search process) 
for this equal cost case. The Shanr 
the nonequal cost case by Blachmai 
in which the choice at each stage may, for example, take different 

times or cost different amounts as in the above diagnostic counterexample. 
The Blachman technique was improved by Marcus ^* using some of 
Huffman's results. A further extension was developed by Karp i^/ 
for which the symbols are not necessarily equally probable and the costs 
not necessarily equal. In his development, Karp uses an integer pro- 
gramming algorithm. Also the codes of interest all have the prefix pro- 
perty, i.e. , it is not possible to obtain another member of the code 
group by adding digits to any given member of the code group. 

Radar Search Processes and Reconnaissance 

3.25 A large body of literature is devoted to the applications of in- 
formation theory to radar detection problems, including both single-stage 
and sequential detection. Such analyses are not examined here since they 
are outside the scope of the sequential search processes defined in this 
investigation. However, the distinction is sometimes difficult to discern 
and a selected number of such algorithms are examined for general back- 
ground and perspective. 

3.26 In 1964, Machol -=-^ examined information-theoretic and other 
limitations on the operation of a radar system. However, his paper 



35 



stresses fundamental limitations rather than explicit search algorithms. 
Danskin i-§/ also uses an information-theoretic measure of performance in 
examining reconnaissance, but he does not set forth explicit sequential 
search procedures. 

19/ 

3.27 Novosad — f examines search problems and information theory 

largely in the context of a radar search for a missile. He considers two 
criteria: 

a. Maximizing the probability of detection and 

b. Minimizing the uncertainty that can be ex- 
pected at the conclusion of the search. 

He shows that if the entropy is used as a measure of uncertainty (item b), 
the two criteria above can lead to different search strategies. In item a, 
one optimum strategy divides the number of examinations equally between 
two search areas. In item b, all the search effort is devoted to one area. 

MINIMUM EFFORT ALGORITHMS 

3.28 Many sequential search processes are posed in generalized terms 
without emphasis on particular applications. It is useful to group the al- 
gorithms developed in these cases in terms of the criteria that are used 

to establish the search strategy. Most of these investigations deal with 
the criterion of minimizing expected effort, measured by cost, time, or 
some function of effort. Note also that several of the information-theoretic 
approaches utilized the criterion of minimizing the expected number of 
steps in the process, or the expected length of a code, which are entirely 
equivalent. The most generalized approach was Karp's algorithm, which 
used integer programming techniques for solution. 

3.29 Two related papers 2 ° ' 2 =/ deal with a search problem that is 
historically significant in that they were motivated by Koopman's work °* 22 / 
on search, which is more appropriately an allocation of resources problem. 
However, in Blachman's work, an algorithm is developed which minimizes 
the expected delay between the appearance and detection of an object, 
where the time of appearance is distributed uniformly and the probability 

of appearance in the ith location is p.. A conventional Lagrange multiplier 
approach is used. In the second paper by Blachman and Proschan «/ 
the object is to maximize a gain function that is nonincreasing in the de- 
lay between arrival and the beginning of the detecting look. In this case, 
objects arrive in accord with a Poisson process. Here again, conventional 
but complicated maximization-of-functions techniques are used to find a 
solution. 
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3.30 In a 1963 paper by Dobbie— A a sequential approach to search 
theory is presented that is also based on the work of Koopman -2r and 
others SttZP/ . Dobbie considers two criteria: minimizing the expected 
effort to attain a given probability of detection and maximizing the de- 
tection probability with a given effort- He concludes that 

" . . .the expected effort is minimized by always dis- 
tributing the effort to maximize the probability of de- 
tection with the effort expended thus far. This is a 
distribution requirement that is very difficult to meet. 
By contrast, the distribution that maximizes the 
probability of detection with a given effort can be non- 
optimal for all values of the effort less than the total 
effort; the effort can be applied by any schedule that 
finally attains the required distribution when all the 
effort has been applied." 

In developing his solutions, Dobbie uses the principle of optimality of 

dynamic programming. 

3.31 Several studies conducted by MIT students have been di- 

rected toward minimum expected cost search processes. In the 

1962 study ^/it is shown that several criteria of optimality lead to the 
same search policy. Pollock's work- — ^ the most extensive and most 
recent, involves a stationary target and a noisy measurement device 
such that probability density functions can be established for cases in 
which a target either is or is not present. A modified Wald sequential 
probability ratio test is employed and a search strategy developed by 
solving a functional equation of the dynamic programming type. This 
is another example in which mathematical programming was the basic 
algorithm used to solve a search problem in which minimization of 
expected effort was the criterion. 

Minimum Effort Diagnostic Algorithms 

3.32 Several algorithms have been developed concerning diagnostic 
routines based on minimization of effort. Gluss JL2/ considers policies 
that minimize the expected amount of time consumed and penalties paid, 
and his analysis is based on dynamic programming concepts. Several 
years later^s Chew considered a diagnostic routine (searching plan) for 
minimizing the expected time or cost. Such a plan "instructs the searcher 
to inspect at each stage the box in which the object is most likely to 

be found." His basic approach is Bayesian and his work is shown to 

be an extension of other allocation of effort and search algorithms. °/ 3 2/3y 
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Minimum Time Radar Search 

34/ 

3.33 Posner — has considered a radar search for a satellite lost in 

a region of the sky and is concerned with minimizing the expected search 
time. A preliminary and final search are postulated, with the former re- 
sulting in a ranking of various portions of the sky and the latter examining 
the regions of greatest likelihood. A preliminary narrow beam search is 
proposed as best. 

3.34 This is the case of a noisy measurement device whose charac- 
teristics change with the allocation of effort to each search cell. Results 
are obtained for the two-stage case (preliminary and final search) and 
extended to the multistage case. Reference is also made to a paper in 
preparation which proposes the following optimal strategy for minimizing 
expected search time: search the most likely cell until it is no longer 
the most likely; then search the cell that has become the most likely. 
Note that Chew's diagnostic routine ^-=/ sets forth a similar search 
strategy. 

HUMAN SEARCH AND DECISION PROCESSES 

3.35 Part of the motivation for this investigation is to obtain a clearer 
understanding of the manner in which a person carries out search and 
decision processes. The search algorithms discussed previously are 
tools that would enable the rational human to implement a search in an 
optimal manner. However, as an imperfect collector, observer, and pro- 
cessor of information, these so-called optimal policies are rarely imple- 
mented, particularly when such operations take place under stress. 

3.36 Since the literature on this subject is massive, the intent was 
simply to provide the serious investigator of human search and decision 
processes with a concise characterization of those processes and algo- 
rithms and a sounder foundation for further theoretical or experimental work. 
Further, it was intended that such a characterization would place in evi- 
dence some obvious missing links through an analysis of the morphologi- 
cal structure of these processes. 

3.37 The studies that have been examined are cited in the biblio- 
graphy. They are quite diverse and deal with all aspects of the cate- 
gories of Figure 1, particularly the human as a measurement device and 
the problem of identifying criteria under which the human operates. 
Appendix II contains some analyses carried out in response to specific 
queries by the Air Force concerning scoring systems that tend to force a 
human subject to act in accord with his subjective probability estimates. 
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3.38 A good example of the relationship between the previous charact- 
erization of search processes and human decision processes is a study of 
a multistage decision task by Rapoport . In this investigation, a brief 
classification of decision theory is provided together with a model for a 
particular multistage decision task. 

3.39 'The model is set up such that the solution, in terms of an opti- 
mal set of sequential decisions, is provided by existing dynamic pro- 
gramming methods. Experiments are then carried out with human subjects, 
the results of which are compared with the optimal solution. 

3.40 What is significant about Rapoport 1 s study in this investigation 
is that it is closely allied to sequential search processes and demon- 
strates the heuristic methods that are used by humans for moderately 
large or complicated decision trees. Interpreted somewhat differently, 
given enough time and technical expertise, the optimal solution to the 
posed multistage decision process could have been found by the subjects. 
In the absence of this, heuristic methods were adopted that provided re- 
sults approximating this optimum in varying degrees. For the multidimen- 
sional search and decision process (Section II) for which no solutions 
exist at this time, heuristic methods are also used to obtain approximate 
solutions. It is also significant that dynamic programming provides a 
firm theoretical foundation for the solution to a broad class of sequential 
search problems . 

SUMMARY 

3.41 This section has provided a brief review and interpretation of 
sequential search algorithms that have been examined. The process of 
sorting and characterizing these algorithms was also discussed. Sec- 
tion IV provides tentative conclusions and recommendations, the results 
of this investigation. A brief description of some useful techniques for 
solving sequential search processes is provided in Appendix I. Appendix 
II presents an analysis of scoring systems as specifically requested 

by ESD. 
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SECTION IV 

SUMMARY 



4.1 This section summarizes the results of this investigation of 
sequential search processes in the form of conclusions, followed by 
recommendations for continued analyses and future applications that 
appear to be of significance to the Air Force. 

CONCLUSIONS 

4.2 Sequential search processes can be characterized as decision 
trees with various structures and properties. The utility of sequential 
search algorithms depends on the structure, properties, and constraints 
they impose on the tree as well as externally imposed measures of ef- 
fectiveness and criteria (cf. Figure 1). 

4.3 A particular search algorithm is the means by which one or 
more "best" paths in the decision tree are chosen, although the reali- 
zation of that algorithm may itself be represented by a decision tree. 

4.4 A broad class of sequential search algorithms may be effectively 
characterized by Figure 1 and the tentative categories in Section III. For 
purposes of further applications, with perhaps some modification and ex- 
tension, this is an adequate and concise characterization of such al- 
gorithms . 

4.5 An exhaustive search through a decision tree is impossible in 
most practical problems because of the enormous number of possible 
paths. Computationally tractible techniques do not exist for handling 
even moderately unconstrained decision tress. In attempting to simplify 
a problem so that is is computationally feasible, it is usually more ef- 
fective to reduce the number of decisions to be made rather than the num- 
ber of available alternatives at each decision point. 

4.6 Any explicit coding procedure, developed from information- 
theoretic considerations or otherwise, can be put into a one-to-one cor- 
respondence with a decision tree. Algorithms for the formal development 
of codes, therefore, may be used to establish search strategies. 

4.7 Aside from the area of coding and selected other applications 
that can be related to coding (e.g. , the coin weighing problem), the 
utility of information-theoretic approaches to sequential search has 
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been limited to characterizing and interpreting such processes rather 
than finding explicit algorithms for their solution. 

4.8 The most significant advances in the development of sequential 
search algorithms have been based on conventional decision theory and 
mathematical programming. The technique of dynamic programming has 
been employed with particular success. (Additional details are pre- 
sented in Appendix I.) 

4.9 The search criterion most widely accepted is minimization of 
expected effort, where such effort is defined in terms such as cost, 
time/ and number of stages. The reason for this is twofold: the mathe- 
matics of sequential search processes is generally most tractible for 
this case and, on independent grounds , there is reason to believe that 
the human earring out such a process most often considers this criterion 
to be most important. Another significant criterion is that of maximizing 
detection probability, which is often used in conjunction with a fixed 
limitation on total available search effort. Algorithms based on these 
criteria are available and have been applied in the areas of coding 
diagnostics, radar search, sequential detection, and warfare . 

RECOMMENDATIONS 

Completion of Present Investigation 

4.10 Further activities recommended within the present scope of 
this investigation are: 

a. Completion of the algorithm coding and subsequent 
sorting and interpretation by similar characteristics 
to provide a compendium of such algorithms and a 
mechanism for their retrieval 

b. Establishment of an explicit correspondence 
between the formal structure of search and de- 
cision trees and applicable algorithms. 

Future Activities 

4.11 Further investigations of the structure and taxonomy should be 
carried out as an adjunct to specific applications of sequential search 
algorithms to problems confronting the Air Force at this time. Within 
the context of these applications, further developments should empha- 
size: 

a. The nature of the constraints most often imposed on 
sequential search processes, e.g., maximizing 

41 



expected effort, probability of detection, 
and stationarity of the search object 

b. The use of the most applicable algorithms (e.g. , 
dynamic programming) to obtain greater search 
efficiencies 

c . Standardization of the pres entation format of the 
sequential search processes and algorithms 
into a handbook to facilitate their application. 

4.12 More explicitly, the areas in which sequential algorithms can 

most effectively be applied to Air Force activities are command and con- 
trol, computation, surveillance, reconnaissance, communications, and 
human decision processes. Specific subareas to which the theory is 
applicable include the search for: 

a. Aircraft or missiles by a radar system 

b. Fixed or moving installations or platforms from 
an aircraft or satellite 

c. Documents or files in a large-scale information 
retrieval system 

d. Data bits in a generalized memory bank 

e. Desired signals in an interference background 

f . Malfunctions in an electronic system 

g. Data, information, or signals by humans 

h. Unoccupied lines in a communications 
switching system 

i. Patterns on a surface. 
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APPENDIX I 



SELECTED MATHEMATICAL TECHNIQUES 



LINEAR PROGRAMMING 



1. 1 Linear programming is a technique for finding the values for the 

. , 2 , . . . , N to minimize 

N 

f (x l7 Xg, . . .x ) = ) c.x (1. 1) 

n z_i li 



variables x. , i = 1,2,..., N to minimize 

N 



i=l 



subject to the constraints x. = o, all Land 
N 



Y a., x. = b., j = 



= 1,2,... M 



(1.2) 



i=l 



with N > M (i.e. , more variables than constraints) and the c. , b. , and 
a., are given constants. The function f is called the objective function. 
Note that if the problem were to maximize 

N 

g(x x , Xa, . . . , x ) = V d x (1.3) 

n z_i i i 

i=l 

this could be brought into the minimization format by seeking the minimum 
value for 



h (xj, Xp , . . . , x ) — 

n 



(-d.)x. 
l l 



(1.4) 



i=l 



1.2 Since, unlike dynamic programming, linear programming requires a 

particular format, there exist computer programs to find these solutions. 
These programs can handle problems with several thousand variables, with 
several hundred constraint equations, not counting the positivity con- 
straints. Since these programs exist, the details of the computational 
techniques would be of interest mainly to computer specialists. 
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However, a brief explanation of the general ideas will be given. In order 
not to burden this discussion with much cumbersome notation, let us take up 
the case where there are 10 variables and 6 constraints. The 10 numbers 
to be found will be considered as a point in a 10-dimensional space; this 
will be done merely to allow a language simplification. 

1.3 It can be shown that if there is a unique, finite solution, then 
exactly 6 of the coordinates will be nonzero. Also, even if the solution 
is not unique, it will be attained at points whose coordinates satisfy 
these conditions, and every solution point can be expressed as a convex 
linear combination of points from this set. Thus, the solution technique 
searches through points with exactly 6 nonzero coordinates, satisfying 
the constraints. 

1.4 The search starts with a first point, satisfying the constraints 
and having exactly 6 nonzero coordinates. There are many techniques 
for finding such a point; let us assume that one has been found. The 
program evaluates the objective function at this point, and then looks 
over the "neighboring" points (this term will be defined presently) to 
see if it can find one that satisfies the constraints , and still get a smaller 
value for the objective function. If no such point is found, it can be 
shown that the present point is the optimal one. If such a point is found, 
a search is made of its neighboring points, under the same conditions. 
Note that, once a point is left, it can never be returned to, since the 
value for the objective function must be less than the current best value. 

1.5 The initial point contained 6 nonzero coordinates and satisfied 
the constraints. It can be shown that this is the only point satisfying the 
constraints and having nonzero coordinates in these positions. A neigh- 
boring point is a point also having exactly 6 nonzero coordinates, of 
which exactly 5 occur in the same positions as in the original point. 
Thus, if the original point is 

(Xx, O, 0,X4,0,X6,X 7 ,Xe,X9,0) (1.5) 

a neighboring point would be 

(o,y 2 ,o,y4,o,y 6 ,y 7 ,Ye,Y9 ,o) (1.6) 

Note that the values in the common positions are not assumed to be the 
same; instead, the property of having a nonzero entry remains the same 
in five of the positions. Again, there will be only one point satisfying 
the constraints and having nonzero coordinates in these particular positions. 
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1.6 The program thus proceeds from the initial point to a neighboring 
point and continues through other neighbors, until the optimal point is 
found. Computational experience has shown that the number of such steps, 
or "iterations" as they are called in the field of linear programming, is 
always less than 3 times the number of constraining equations. 

1.7 Whereas this technique will solve any type of linear programming 
problem, certain special types of problems are solved by special tech- 
niques that work for larger numbers of variables or constraints, or obtain 
solutions in a shorter period of computer time. The so-called "Transpor- 
tation Problem" falls into this category. 

BACKTRACK PROGRAMMING 



1.8 "Backtrack programming" is a method for finding the best path 
through a decision tree with a separable objective function which, under 
certain conditions, will involve a less-than-exhaustive search. Like 
dynamic programming, and unlike linear programming, it is more a phi- 
losophy than a definite procedure. Basically it consists of a bookkeeping 
procedure which omits examining certain paths through the tree when it 
can be seen, from prior knowledge, that they will not be optimum ^ There 
seems to be only one paper in the literature that deals with this, — and 
in this paper is the statement "Backtrack is no more than an educated 
exhaustive search procedure." 

1.9 To illustrate by means of an example, suppose we are trying to 
find nonnegative integer values for the variables x a ,x$ , . . .x*, subject 
to the constraint 



8 

i 

i=l 



x. = M 

l 



(1.7) 



for some known constant M, such that 

8 



i=l 



(1.8) 



where T is a given constant, f.(x.) are known functions, and i § (x t ) = 

i i i i 

for all f and for all allowable values for each x. . 
i l 
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1. 10 Now suppose a search is started by fixing a value for x x and 
trying all values for Xq with this. To be more particular, suppose we are 
trying to find a solution with x 1 = 3. Then, in trying to find a value for 
Xs , the integers to try for this variable are those between zero and M-3. 
Suppose that, for all of these values for Xg, 

fi 0)+ f 2 (*)> T (1.9) 

1. 11 Then it is obvious that no solution can be found using x x = 3 
since the f,(x.) are all nonnegative. Henc the number of possibilities 
has been reduced. 

1.12 As the paper points out, some sort of balance must be achieved 
between the. search procedure and the bookkeeping procedure, because if 
the bookkeeping gets too complicated it becomes more time-consuming 
than the straight exhaustive technique. 

DYNAMIC PROGRAMMING 

1.13 "Dynamic programming" is a way of looking at problems of maxi- 
mizing or minimizing functions of several variables, satisfying constraints, 
that sometimes reduces an N-dimensional problem to a sequence of one- 
dimensional problems. 

1.14 As an example of this philosophy (because, unlike linear Pro- 
gramming, it is not a well-defined technique), let us look at the following 
problem: Maximize the product of N variables subject to the constraints 
that all N variables be positive and their sum be less than or equal to a 
fixed amount B. In mathematical notation, this can be written: 

N 
Maximize F (x i# Xg ...x )= IT x. (1. 10) 

i= 1 

N 

> V < 

subject to x. = o, all i, and ) x. = B (1. 1 1) 

1 Z_i 1 

i= 1 

1.15 This is a generalization of the elementary calculus problem of 
finding the rectangle of maximum area with given fixed perimeter. If 

N = 2, then x x = B/2, Xg = B/2 gives the maximum value. Note that this 
solution did not depend on the particular value of B. No matter how much 
is available, the best way to solve the two-variable problem is to divide 
the available amount into two parts. 
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1. 16 Suppose we look next at the three -variable problem. For any 

choice of a value for the third variable, we know how to proceed to 
maximize the product of the other two. Thus, if we allocate an amount 
y for the third variable, there will be B-y remaining for the other twp, 
and the best way to divide this amount among them is to split it equally, 
with (B-y)/2 for each variable. For each choice for the third variable, then, 
the conditional maximum value (being dependent on the value for y) is 
given by 



G 3 (y) = y 



B-y 



*ZZ. 



(1.12) 



1. 17 The problem is: What would be the best value for y? But using 

the standard method of elementary calculus, we solve the equation 



so that 



Gk(y)= 






B 3 - 4By + 3y 3 



= 



(1.13) 



(1.14) 



This gives y = B/3 and y = B. However, this second value gives a 
minimum. Thus X3 = B/3, leaving x 1 = Xg = 1/2 (B - B/3) = B/3. Again, 
the answer to the three-variable problem can be expressed as a policy: 
Divide the available amount into three equal parts. 

1.18 Continuing one more step, let us try the four-variable problem. 

Again here, for every choice of a value for the fourth variable, we know 
how to proceed with the other three to maximize their product. Thus, for 
every choice y for a value for x*, if we continue by making the "best" 
allocation of the rest, we get 



G* (y) = y 



•Ct*) 



(1.15) 



1.19 Again, finding the "best best" by ordinary calculus, we find 

y = B/4, so the optimal allocation for the four-variable problem is 



Xx = Xe =X3 = X4 = B/4 



(1.16) 



1.20 If we wanted to continue this way, we could solve for N = 5,6. 

etc. , each time using the previous solution to solve the new problem. 
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Thus we have replaced the original problem of maximizing a function of 
N variables by a sequence of N-l problems, each involving the maxi- 
mizing of a function of one variable. (In this particularly simple prob- 
lem, it should be evident that the optimal solution for the problem with 
K variables is to allocate B/K to each variable; this can be proved by 
using mathematical induction.) 

1.21 Let us examine the qualities of this problem that enabled us 

to apply this technique for solution. The key feature was that the function 
to be maximized in the problem with K variables contained the function 
used for the problem with K-l variables (with the constraints changed), 
and this was the only place the first K-l variables appeared. In a more 
mathematical notation, if F (x x , x 2 . . .x ) is the function to be maximized 
in the problem with K variables, and F -l {x 1 t Xs, x ) was the corres- 
ponding function for K-l variables, then 

F K (xi, x 2 , .x K ) = G Ff k _ 1 (x if Xg, . x^), x K J (1.17) 

1.22 F had to be a nondecreasing function of F , since when we 
wanted to find the conditional maximum for F , we made F as large 
as possible. We also needed to be able to express the maximum value 
of F , for any value of x that satisfied the constraints, as a function 
of x . Thus the solution 01 the problem of maximizing F must be a 
"poncy," that is, a function of a parameter whose value can range from 

to B. In the more general case where the constraint region is more com- 
plicated than the interval that was used in this model problem, for 
every value of x such that there are points in the constraint region having 
this number for their Kth coordinate, the optimal value for F over that 
set of points must be known. 

1.23 For the technique to be practical, then, the optimal policy for 
F should be expressible explicitly as a function of x . For instance, 
ir the optimal policy for F is known only to the extent that it is ob- 
tained by the simultaneous solution of a set of nonlinear equations in- 
volving the x, , then the ensuing one-dimensional maximization problem 
may lead to grave computational difficulties. (In fact, the solutions may 
not even be continuous functions of the parameter.) 

1.24 The same sort of ideas may be applied to problems in which only 
a discrete set of values is available for the variables. An example of 
this could arise in a problem in allocation of resources with a fixed 
budget, in which the resources come in unit size. To simplify our 
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problem, assume that the costs per unit of the resources also are integers. 
(This problem is based on one given in "Nonlinear and Dynamic Program- 
ming" by Hadley.^-^ Suppose that the return from using x units of re- 
source i is given by a known function f (x ). Then the problem is 



Maximize 



F(x X/ Xe , . --x N ) ■ 2, W 

i = 1 



(1.18) 



subject to the constraints N 

x. > o, ) a, x. < 
i = Z-i 11 = 

i=l 



B 



(1.19) 



where the a , x. , and B are all nonnegative integers . 

1 N 

1.25 The expression L, a, x., for any possible choice of K and 

the x./Will always take on only nonnegative integer values, since both 

the a, and the x. are integers. 
i i 

1.26 The philosophy for solving this problem is the discrete analog 

to that used in the continuous case. Suppose we have found the optimum 

policy for the problem involving K-l variables, that is, for every integer 

M less than or equal to B we know the optimum allocation for x 1 ,x^, . . . 

x , subject to 
K— 1 



K-l 



i i 



M 



(1.20) 



i=l 



We can think of these results as being kept in a table, and now we 
desire to make up a similar table for the sum of the first K functions. 
To do this, let us see how we would make up the entry in the new table 
for the integer M . Thus we want to find the maximum value for 



W 



(1.21) 



i=l 



subject to 



K 



) a.x, < M 
L> , i l - o 



(1.22) 



all the other constraints holding as before, 
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1.27 Thus we have the problem of allocating M among x x , Xq . . .x 

in an optimal fashion. To do this, we look at all tne possible ways 01 

allocating part of the M to x and the rest to the other variables. So, 

suppose we want to see what would happen if we allocated x8 for x . 

(Of course, we must have a x°< M .) 

K K - o 

1.28 Then there is available M - a x° for the variables x 1# Xq, . . . 

x . But the table for F already lists the best allocation for these 

variables, given (M - a Tjr x° ), since it lists the best for every integer 

o K K 
up to B, and M - a Tjr x°< B. 
O K K- 

1.29 Now, with this allocation of x° we can compute max F__(x ). 



thi 



Whichever value for x that maximizes this is the best amount 

for x ; and this value, together with the values for Xi, Xg . . .x just 

obtained, is the value for this particular way of splitting up M . 
i o 

1.30 Using this method, letting x° successively equal 0, 1, 2, . . . 

up to the biggest possible value that x can take subject to a x°< M , 

K K K — o 

and then letting (M -a x°) be available to Xi, x@ , , . .x we pick 

O K K K~ 1 

the allocation that gives the biggest value— that is, the "best best." 

This becomes the table entry opposite M in the table for the sum of 

the first K variables. 

1.31 Since it is evident how the table should be made up for x x all 
by itself, and since the method just outlined shows how to get the next 
table given the table previous, the table for any number of variables can 
be obtained. 

1.32 Although this method looks complicated, and indeed a computing 
machine should be used for most problems, it can be shown that it is much 
more efficient than a straightforward exhaustion method. 

COMBINING PROBABILISTIC AND DETERMINISTIC EVENTS 

1.33 To illustrate the problems involved and the various choices of 
criteria possible, let us set up a model problem. Suppose that there 
are two urns on a shelf. Urn number one we know to contain 99 red 
marbles and 1 blue, while urn number two contains 60 green marbles and 
40 yellow. Knowing the composition of each of these urns, you choose 
the one you want and then choose, without looking, a marble from that 
urn. A sum of money is awarded you, depending on the color of the 
marble drawn, in accordance with the following scheme: 
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Marble Color 


Payoff 


Red 
Blue 
Green 
Yellow 


10 

1000 

20 

25 



1.34 The total process can be illustrated by a tree, in which a 

triangular box indicates a chance event. Figure 13 illustrates this. 
The numbers alongside the lines leading from the triangular boxes 
indicate the probabilities of taking those paths, and the numbers in the 
square boxes at the end give the values for these paths. 







FIGURE 13. SIMPLE DECISION TREE 
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1.35 In this case, the decision-maker must decide whether he pre- 
fers urn one, which gives him a 99-percent chance of gaining $10 and a 
1-percent chance of gaining $1000, to urn two, which gives him a 60- 
percent chance of gaining $20 and a 40-percent chance of gaining $25. 

1.36 One commonly used method for making this choice is to take 

the path that offers the largest expected value. In this case, the expected 
value for urn one is 

go l 

E [1] =JooT * 10+ loo" " 1000= 9 - 90+ 10 - 00 = !9-90 (1.23) 

while the expected value for urn two is 

E [2] = -^T ' 20 + -~- • 25 = 1.00+ 10.00 = 22.00 (1.24) 

So, if this criterion is used, urn two will be chosen. 

1.37 The reasoning behind the use of this criterion is the theorem 
that says if a probability experiment is performed N times, where N is 

a large number, the total return from these N trials will tend to be rela- 
tively close to NE, where E is the expected value of a single trial. 
Here we mean by "relatively close" that the difference between the 
actual sum and the quantity NE, when divided by NE, will tend to be 
close to zero. The larger N is, the more confident we are that this 
quotient will be small in magnitude. Thus, in this particular case, for 
urn one we should expect the total return after N trials to be close to 
19.90 N, whereas for urn two we would expect the total to be close to 
22.00 N. But, if the experiment is to be done only once, neither the 
result $19.90 nor $22.00 is possible. Thus, although this criterion 
may be quite useful when we are picking a long-range policy for an 
experiment to be performed many times, its usefulness when the choice 
is to be made only once is not so apparent. 

1.38 Another possible method of making this decision is to set some 
level of return as being the dividing point between satisfactory and un- 
satisfactory, and then pick the experiment that gives the highest prob- 
ability of attaining the satisfactory level. This is equivalent to replacing 
all the original values of the results by zeros and ones, where original 
values over or at the acceptance level are replaced by ones, and those 
under by zeros, then applying the expected value criterion to this zero- 
one problem. Thus, in this problem, suppose the acceptable level were 
set at $23. Then urn two would be chosen. However, if it were set 
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at anything above $25, urn one would be chosen. This lack of continuity 
in the choice process might leave one a little uncomfortable about using 
it. 

1.39 Another possibility would be to use some combination of these two 
methods. Thus, one could set a level below which returns would be worthless; 
returns above this level would have a value proportional to the difference 
between them and the acceptance level. Then, the expected value criter- 
ion could be used on the problem with this new set of payoffs. A closer 
look at this scheme indicates that the only real change from the original 
expected value scheme has been in the valuation of the possible results. 
In fact, most other methods for making this choice seem to involve noth- 
ing more than a change in the valuation of the payoffs, followed by the 
use of the expected value criterion on the new problem. 

1.40 One important point is that in trees with probabilistic choices, 
certain criteria which were meaningful in the completely deterministic 
case are not meaningful. For instance, the criterion "find the path yield- 
ing the largest return 11 is certainly applicable in the completely deter- 
ministic case, but it is meaningless in the probabilistic case. 

1.41 Finally, also note that the schemes for combining probabilities 
and outcomes that were outlined here are also applicable to problems in 
which more than one probabilistic stage is reached in the problem. The 
size of the tree involved in such a problem makes it difficult to draw one 
here, but a verbal explanation may help. Suppose that initially one of two 
paths must be chosen, and that each of these paths leads to two possible 
nodes, the node being chosen randomly, but with known probability. 
Now, from each of these four nodes, there are two choices, each of which 
gives rise to two possibilities with known probabilities, and the value 

of the final payoff for each of these 16 results is known. 

T.42 In this problem there are two choices to be made: one initial 

choice and one after the first chance event. A strategy consists of a 
choice at the first level plus a scheme for making a second-level choice 
to go with each possible outcome from the first-level probabilistic event. 
With each of these strategies, however, the probability of attaining each 
of the payoff boxes is easily computed, so that any scheme for rating 
strategies by combining probabilistic and deterministic events is appli- 
cable . 

Extension to Noisy Measurement or Detection Devices 

1.43 As discussed earlier, in a stochastic decision process the 

decision-maker chooses from among a class of probability distributions. 
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FIGURE 14. DECISION TREE ILLUSTRATING 
NOISY DETECTION DEVICE 
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A sample is then obtained from the distribution of his choice, and, having 
observed this sample, he then makes a choice from the class of distribu- 
tions available at the next level, continuing this way until the final step 
in the process is reached. The objective, of course, is to find the best 
decision rule; i.e., he should have a first choice, and then, for each 
possible outcome of that first choice, a second choice, etc. 

1.44 Suppose now that, instead of the decision-maker observing di- 
rectly the samples from the probability distributions, the result is reported 
to him by a device that can be inaccurate, e.g. , a radar; this will be re- 
ferred to as a "noisy detection device." We assume, of course, that 

for each possible sample from each of the available distributions, the 
probability of each possible report is known. We will show that this 
extra complication can still be handled within the context of a stochastic 
decision tree. 

1.45 To that end, let us examine the simplest possible problem to 
which this complication can apply. The extension of the idea to more 
complex problems should be evident. We will initially look at the 
problem as it would be with a perfect reporting device and then see what 
changes must be made if the device has the possibility of reporting in- 
correctly. 

1.46 In this problem, there are two distributions to choose from at 
the first step. Each distribution has just two possible outcomes. Thus 
there are four possible starting points for the next level. Each of these, 
in turn, gives a choice from two distributions, with two possible samples 
from each distribution. Since the problem ends at this level, there are 
16 possible ending points. Figure 14 is the portion of the decision tree 
ensuing from the choice of the left branch at the first decision level; 

the tree following the choice of the right branch is identical in structure, 
but of course the probabilities and payoffs may be different. 

1.47 Now, suppose that the detection device is noisy. This means 
that when the device reports that we are at 2, there is a certain (known) 
probability that we actually are at 2, and also a (known) probability 
that we are at 3, with analogous statements applying in the other case. 
The courses of action available from node 2 and node 3 must be identical 
(although the probabilities of the chance events and the payoffs may be 
different); this follows from the observation that, if there were different 
courses of action available, then, for instance, if the decision-maker 
mistakenly thought he was at node 2, when he actually was at node 3, 
and if he tried to take a course of action not available at node 3, he 
would find this out and realize that he was at node 2. 
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1.48 So, if the decision-maker chooses "left" at the second level, he 
does not know whether he is choosing the distribution that follows the 
left choice at node 2, with outcomes 4 and 5, or the distribution that fol- 
lows the left choice at node 3, with outcomes 6 and 7. But, he does know 
the probabilities that he is at node 2 and node 3 and, for each of these 
events, the probabilities of the various outcomes. Then, for instance, if 
the report on the outcome of the first choice is that he is at node 2, and 

if he chooses "left," there are four possible outcomes, and their proba- 
bilities are known. The possible outcomes are numbers 4,5,7, and 8, with 
the probability of obtaining 4 being the probability that he is at 2 multiplied 
by the probability of obtaining 4 given that he is at 2; similar calculations 
can be made for other possibilities. Then, the original tree can be re- 
placed by one in which the possibilities at the second decision level rep- 
resent reports of the noisy detection device and the probabilistic outcomes 
of the next choice are replaced by the more complicated ones obtained by 
combining the uncertainty of where you are in the original tree with that of 
what the outcome of your choice will be. 

1.49 Finally, if the probabilities of the false reports are given in the 
form of the probability that the report will be node "i" given that you are 
at node "j," then Bayes' Rule must be used to obtain the probabilities that 
you actually are at node "i M and node "j." 

INFORMATION THEORETIC SEARCH 

1.50 An example of the utility of information theoretic methods with * 
respect to sequential search can be found in a coin weighing problem. — 
Assume that there are 12 coins, of which 11 are true coins and one is false. 
The false coin may be either lighter or heavier than the others. It is de- 
sired to find the false coin and determine whether it is lighter or heavier. 
The available measurement device is a noiseless equal arm balance. 

1.51 The uncertainty to be resolved is of measure log 24 since each 
of the 12 coins may be false and lighter or heavier. It is assumed that 
all the possibilities are equally likely (prior distribution is uniform). 
With three weighings, the maximum information that can be obtained is 

3 log 3 = log 27> log 24 
Therefore it may be possible to completely resolve the uncertainty. 

1.52 If in the first weighing i coins are placed on each pan, then the 
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TABLE I. SECOND WEIGHING ALTERNATIVES, 
PROBABILITIES AND ENTROPIES 



Index 

> 


Probability 


Entropy, 
H (dits)^ 


j. 


J 


P(B)^ 


P(R)^ 


P(LJ^ 


i 


1 


1 


1 
4 


4 


0.452 


i 





4 

4 


t 


1 
3 


0.320 


2 


2 





* 


* 


0.301 


2 


1 


1 

4 


3 
8 


3 

8 


0.470 


2 





i 


i 

4 


4 


0.452 


3 


1 





£ 


1 


0.301 


3 





4 


1 


3 

8 


0.470 


4 








1 


i 


0.301 




^ P(B) 


= probability of balance. 






V P(R) 


= probability right side heavier. 






^ P(L) 


= probability left side heavier. 






^ H-. 


sntropy = - Ep.log p. . 
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respective probabilities of balance, right side heavier or left side heavier 
are: 

6 - i 

Prob (balance) = ; (1.25) 

o 

Prob (right side heavier) - — = Prob (left side heavier) (1.26) 

1.53 To force these probabilities to be equal and thus obtain the maxi- 
mum amount of information (log 3) on the first weighing, this value of i is 
chosen to be four. Now the pans may or may not balance. Consider first 
the case in which they do balance so that the false coin is known to be 
among those that were not used in the first weighing. 

1.54 On the second weighing, place i of the four suspected coins in 
the righHiand pan and j ^ i of these in the left-hand pan together with 
i-j of the true coins. Table I shows the various possibilities for values 
of i and j, the probabilities, and the overall entropy. 

1.55 Since the entropy is greatest for (i,j) = (2,1) and (3,0), it is 
reasonable to consider the results of such choices. That is, the greatest 
resolution of uncertainty is obtained with either of these two measurements, 
Using the measurement (i,j) = (2,1), the symbolic representation of such 

a weighing will be (Si, Sis ) vs (S 3 ,T), where Si, Sis , an d S3 are three of 
the suspected false coins and T is one of the known true coins. If bal- 
ance is achieved then the false coin must be S4. Its relative weight 
(heavier or lighter) can then be ascertained on the third weighing by a 
measurement against a known true coin, i.e. , S* vs T. If the left side 
(Si , Sis) is heavier, then S4istrue, and either Si or Sis i s false and heavy 
or S3 is false and light. The third measurement is then Si vs Sis- If bal- 
ance is obtained, S 3 is false and light. If balance is not obtained, the 
heavier side has the false coin, which is heavy. 

1.56 For the second measurement denoted by (i, j) = (3,0), a similar 
procedure can be followed. The measurement is represented by (T,T,T) 
vs (Si , Sis , S3). If balance is obtained, S4 is measured against a known 
true coin. If, on the other hand, the right side is heavier, then Si or 
Sis ° r S3 is false and heavy. The third measurement could then be Si vs 
Sz . For balance it is concluded that S3 is false and heavy. Otherwise, 
the heavier side is false and heavy. 

1.57 Thus it is seen that for balance after the first weighing of four 
against four, two alternate schemes are available for the second and third 
weighings, each of which can completely resolve the uncertainty. These 
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TABLE II. SECOND WEIGHING ALTERNATIVES IF BALANCE NOT 
ACHIEVED ON THE FIRST WEIGHING 3/ 



Index 


Probability 


h 


is 


Ji 


J3 


PfB)^ 


P(R>S/ 


P(L)^/ 


2 
2 
2 
1 
1 
1 
3 
2 
2 
2 
1 
3 
2 


1 

1 

1 ' 

2 

2 

2 

1 

2 

2 

2 

3 

2 

3 


2 
2 
1 
1 
1 

1 
1 
1 


1 



1 

1 
2 
1 
2 

1 

1 
1 

1 


k 

f 

* 

f 
I 

1 
f 

3 

8 

j. 
4 

i 
4 


f 
k 
t 

i 
4 

1 
t 
t 

i 

t 

4 

3 

8 

3 
8 


f 
t 

t 

f 

t 
f 
i 

3 

9 

3 

8 

3 
8 


a/ 

H = entropy = - Lp.log Pj, and is equal to 0.47. 

P(B) = probability of balancing. 

c/ 

-^ P(R) = probability right side heavier. 

P(L) = probability left side heavier. 
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acceptable alternates for the second weighing were determined by examin- 
ing the entropies associated with the alternatives and choosing those for 
which the entropy was a maximum. This afforded the maximum resolution 
of uncertainty on the second measurement. The third measurement was 
trivially determined. 

1.58 Consider now the case in which balance is not achieved on the 
first weighing. Assume that such a weighing was (Si, Sb , S3/ S4) vs 

(Ss / S 6/ S7/ Sa) and further that the right side was heavier. (A completely 
analogous argument holds if the left side was heavier.) The conclusion 
that can be drawn is that either one of the four coins (Si, Ss , S3/ S4) is 
false and light or one of the four coins (Ss, Se, S7/ Se ) is false and heavy. 

1.59 On the second weighing consider the placement of i x of S5 / S 6/ S7/ 
Se and i^ of Si , Sg , S3 / S4 in the right-hand pan with j x of Ss , Se/ S 7/ Sa 
and j 3 of Si / $3, S3/ &t in the left-hand pan together with (i x + i a ) - (j x + j a ) 
of the coins that were determined to be true in the first weighing. Further, 
ii + ^ ^ Ji + Js • The total number of admissable possibilities can be limited 
further by noting that the third weighing provides at most log 3 units of in- 
formation. Hence no more than three possibilities may remain after the 
second weighing. Therefore the number of suspected coins not used in the 
second weighing cannot be greater than 3. It is concluded that 

8 - (ii + la + Ji + h ) * 3 
or i x + is + ji + j a * 5 (1.27) 

1.60 Since ii + 1q ^ji + j S / then i x + i s ^3. If the right-hand pan in 
the second weighing is heavier, then either one of the ii coins on the 
right is false and heavy or one of the j a coins in the left is false and light. 
A similar argument can be made for the left hand pan being heavier. For 
these cases , the two further restrictions on the number of possibilities 

on the second weighing are ii + j s ^ 3 and i^ + ji ^ 3 . The remaining possi- 
bilities are now shown in Table II. 

1.61 For each of the 13 alternatives the entropy is the same, i.e. , 
0.47 dits. This may be compared with the maximum of log 3 = 0.477 dits. 
Consider the case (i x , i^, } lt j 2 ) = (2 # l / 2 / l) for which the following meas- 
urement is made: (S?/ S 8 / 5s) vs (Ss, S 6 / Si). It is also recalled that the 
right side was assumed to be heavier on the first weighing, i.e. , (Si, S^ t 
S3, &) vs (Ss, Se, S 7/ Sa). 
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1.62 If balance is obtained, then either S3 or S* is false and light. 
On the third measurement, then, S3 can be weighed against S* and the 
light side is false and a light coin. If, on the second measurement, the 
right side is heavier, then either Ss or Se is false and heavy or Sa is false 
and light. A third measurement of Ss vs Se will resolve this uncertainty. 
If balance is obtained, then 3s is false and light. Otherwise, the heavier 
side is false and heavy. 

1.63 Thus it is shown that the uncertainty is resolvable for the case 
in which balance is not achieved on the first weighing. Furthermore, 13 
alternate schemes for the second weighing are available, all of which 
yield identical resolutions of uncertainty. This indicates that many alter- 
nate strategies are available for solving this problem, which itself is 
only a special case of the set of sequential search algorithms discussed 
in the body of this report. In this special case, we have a multistage pro- 
cess in which the search space is discrete (24 cells), the initial distri- 
bution is assumed to be uniform, the search object is stationary, and the 
measurement device is noisele&s . The partitioning is multicell by virtue of 
the freedom to group coins. The allocation of effort to each cell is, in 
general, variable, and the basic criterion is to carry out the search in no 
more than three steps. Satisfying this criterion is implemented by the 
essentially heuristic method of maximizing the information return per 
stage. 

1.64 If some of these restrictions are removed, such as changing the 
number of false coins to some arbitrary value or introducing noise into the 
measurement device (the equal arm balance) so that it provides a true 
measurement only some percentage of the time, it may not be possible to 
completely resolve the uncertainty. However, as the number of measure- 
ments increases, it may be possible to successively decrease the vari- 
ance of the distribution associated with the search space. This type of 
problem in its fullest generality has not as yet been solved. 



61 






APPENDIX II 
SCORING SYSTEMS 






II. 1 A request was made by ESD to examine the scoring systems de- 

fined in "Admissable Probability Measurement Procedures," by E. H. 
Shuford, Jr. , A. Albert, and H. Edward Massengill, M/ with the objective 
of: 

a. Determining which function of the class 
of derived scoring functions for the binary 
case is preferred and the reason for such 
preference. 

b. Solving the m-nary case in its fullest 
generality. 

This appendix presents the results of investigations to date in each of 
these areas . 

Scoring Functions and the Confounding of the Motivations of Students 

II. 2 In a test, the student is confronted with a question to which he 

replies by wagering r on one possible answer and 1-r on the other (if 
there are only two choices). There is a scoring function, f(r), which then 
gives the student his grade on that question. The fundamental property 
required of scoring functions (by Shuford) is that they be "reproducing;" 
that is, that by choosing an r corresponding to p, his subjective proba- 
bility of the truth of the first alternative, the student maximizes his ex- 
pected score: 

F(p,r) = p f(r) + (1-p) f(l-r) <; p f(p) + (1-p) f(l-p). (II. 1) 
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A way to ensure that f(r) is reproducing is to take 



.; r 



f(r) = \ (l-t)g(t)dt 



g(t) = g(l-t)* 



(II. 2) 






where g(t) is an otherwise arbitrary function, 
take f(l) = 1. 



It is convenient also to 



II. 3 The use of the reproducing property depends on the student being 

motivated to maximize his expected return on the basis of a subjective 
probability. The student may be otherwise motivated. For example, sup- 
pose the student views the test as a game between the tester and himself, 
in which instead of subjective probabilities applying, the tester is select- 
ing values of p to minimize the student's score. The student then selects 
a value for r to maximize his score in face of this kind of opposition. That 
is , he considers the expected value of 






max min F(p,r) 
d(r) p 



(II. 3) 



where d(r) is a distribution of choices of r. On the other hand, the student 
might operate on a minimum regret basis. If he does, he considers the ex- 
pected value of 



min max { p [ 1 - f (r)] + ( 1 - p) [ 1 - f ( 1 - r)] } 
d(r) p 

= min max [ 1 - F(p,r)] . 
d(r) p 



(II. 4) 






Since the roles (maximizing or minimizing) of p and r and the sign of F(p,r) 
are all reversed from those in Equation (II. 3), the strategies for solution 
of this game will be the same as those for the solution of the previous game. 
Whatever f(r) is, these two motivations will be confounded. Therefore the 
minimum regret game can be put aside. But for some choices of f(r), the 
game and expectation motivations are also confounded. 

II. 4 Consider the quadratic scoring function 

f(r) = 2r - r 3 

(II. 5) 

g(t) = 2 
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If the student treats the test as a game with his score as the payoff to him- 
self, his optimum strategy is to take 

r=£ (II. 6) 

and the payoff 

M = p[f&)+ (1 - p)f»)] = f&) = f . (II. 7) 

Optimum tester strategy would be to take 

P=t 
since then 

p f(r) + (1 - p) f(l - r) =£ (1 + 2r - 2r 2 ) ^ | (II. 8) 

although optimum tester strategy is not exactly germane. Thus with this 
scoring function, if a student gives an r = £ answer, it is not clear whether 
he is totally ignorant (p '=£) and maximizing his expectation or is treating 
the test as a mathematical game— to mention only two of the possible moti- 
vations . 

II. 5 The fact that taking r=| is an optimal strategy arises from the 

.reproducing property itself. Since 

F(p,r) = pf(r)+ (1 - p)f(l - r), 

(II. 9) 
F(p,*)«f&) 

independent of p, and this equality, with the inequality (II. 1), assures that 

r =£, p -i 

is a saddle point to the game with F(p,r) as a payoff function. Therefore 
any scoring function that is reproducing will confound the expectation 
and game motivations. 

Forceful Scoring Functions 

II. 6 The question has been raised of how to choose an f(r) so that the 

student is forced, in some sense, to make a correct estimation of his own 
subjective probabilities. It is difficult to see how the idea of making a 
mistake in estimating subjective probabilities is at all tenable. However, 
it is easier to understand a student making a mistake in estimating his 
expected payoff, F(p,r). 
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II. 7 Suppose the student picks 

r = p + c . 



(11.10) 



That is, suppose that the student's error can be characterized by saying 
that he misses finding the maximum value of F by an error E(F). If F(p,r) 
is expanded about r = p, with 



SF(P/P) _ a 
dr 






(11.11) 



then 






F(p,r)« F(p,p)+£ (r-p) 3 — 3 

or 



2 dfF(p,p) 



(II. 12) 



Thus 



E(F) = F(p,r)- F(p,p)«i Ar 



a S 3 F 



(11.13) 



and Ar, the error in hitting the subjective probability, is made small for 
a given E(f) by choosing that function F(p,r) for which 



d 2 F 
dr 



r = p 



= M(p) 



(11.14) 



is as large as possible. 

II. 8 This second derivative is a simple function of the kernel g. 



since 



so that 



M(p) = pf" (p)+ (1 - p)f" (1 - p) 
f" (p)= (1 - p)g' (p) - g (p) 
f" (1 - p) = - pg' (p) - g (p) 

g (1 - p) = g (p), 
- g' (l - p) = g' (p) . 






(11.15) 
(11.16) 
(11.17) 

(11.18) 



65 



Hence, 

M( P ) = p(l-p)g' (p)-pg(p)-p(l-p)g' (p)-(l-p)g(p) = -g(p) . (11.19) 

Thus the "forcing" quality of a scoring function depends just on the kernel. 

II. 9 The kernels for the four scoring functions listed by Shuford are 

as follows : 

a. g(u) = 2 (11.20) 



b ' 9(U) = (l-u)ln2 U< * 



(11.21) 



1 



i < u 



uln2 
g(u) = [u 3 + (1 - u) 3 ]"^ (11.22) 



q(u) = z 5": u < i 

yv ' (1 - u 3 )ln3 * 



(11.23) 



2 



[1 - (1 - u) 2 ]ln3 



h < u . 



All these kernels give concave reproducing scoring systems. These func- 
tions are illustrated in Figure 15. Only half the range of the argument is 
shown since the functions are symmetric. The function a, of course, 
represents the highest value which can be maintained across the whole 
range of the argument. When in the testing procedure there is no interest 
in preferring precision at one level of the student's subjective probability 
p to that at another, this function seems ideal. When there is more in- 
terest in precision at some particular p, the other functions may look in- 
teresting. Functions b, c, and d, however, all put the extra precision in 
the region around p = 0.5. 

II. 10 A function which makes more precision available at extreme 
values of p is that labeled "e" in Figure 15. This function is a member 
of the family 

g(u)= [B (1 - a, 2 - a) [u (1 - u)] a } -1 (11.24) 

where B(x,y) is the complete beta function. If a =^, this yields 

9<u) - ,ju h - ui (II - 25) 
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FIGURE 15. FORCING QUALITY OF SCORING FUNCTIONS 
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More General Scoring Functions 

II. 11 The problem of a general symmetric scoring function can be 
stated as finding a scoring function 

f (Xi , x 3 , . . . , x n ) (11.26) 

that is symmetric in the last n - 1 variables, where 

n 

1 



i*.- 



i=l 
and 



x. ;> . 

i 



The elementary symmetric functions are (x 2 ), (x 2 x 3 ), (x 2 x 3 x 4 ), 
(x 2 x 3 x 4 . . . x ) where 



n' 

n 



(x 2 ) = y x i 

1=2 



n 
(x 2 x 3 ) = V x. x. etc. 

i<J 

there are n - 1 such expressions; however, the first one (x 2 ) = 1 - x a , so 
that it can be suppressed. The scoring function becomes 

f [x 1# (x 2 x 3 ) # 1x2X3X4),. . . (x 2 x 3 ...x n )] . (11.27) 

II. 12 In particular, for n = 3, the scoring function has the form 
f(x # yz). The expected payoff is 

pf(x # yz) + qf(y, zx) + rf(z, xy) . (11.28) 

The problem is to find f such that this expression is maximum for x = p, 
y = q, and z = r # subject to the constraint x + y + z = 1. Following 
Lagrange we add to the expected payoff X (x + y + z) and set all partial 
derivatives to zero and x = p, etc. This yields the following equations. 
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Pfi (P/ qr) + qrf 2 (q, rp) + qrf 2 (r, pq) + \ = 
qi 1 (q, rp) + rpf 2 (r, pq) + rpf 2 (p, qr) + \ = 
rf x (r, pq) + pqf 2 (p, qr) + pqf 2 (q, rp) + \ = 



(11.29) 



These imply that X is a symmetric function of p, q, and r, so that each of 
the other parts of the equation must be symmetric. That is, the requirement 
is that 



pf x (p, qr) + qrf 2 (q, rp) + qrf 2 (r, pq) 



(11.3 0) 



be a symmetric function of p, q, r. 

II. 13 A general "quadratic" scoring function has been found which 
satisfies the added constraint that f(l) = 1, f(0) = 0. It is 

x i [2 - x x + 2(x 2 x 3 ) + 6(x 2 x 3 x 4 ) + . . . + (n- 1) ! (x 2 x 3 . . .x )] . 



(11.31) 
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